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METHOD AND APPARATUS FOR DISPJ-AYING INFORMATION IN A DISPLAY 
SCREEN REGION IDENTIFIED BY PERMANENT PRINTING 

Field of the Invention 
5 Embodiments of the present invention relate to portable computer systems. 

More particularly, embodiments of the present invention provide a portable viewing 
and computing apparatus and method to display information in a display region in 
an area marked by permanent printing that may be used to facilitate gesture 
recognition. 

10 

a 

4i BACKGROUND Art 

?l A conventional hand held computer system is shown in Figure 1 . System 

W 100 contains display unit 110 and permanent printing 120, which is used to 

h) facilitate gesture recognition. There is no display capability in system 100 in the 

CJ 15 region identified by printing 120. 

fU 

rl 

p Display unit 110 and printing 120 are functionally coupled to a digitizing 

panel (not shown) which is responsive to touch across the combined area of 
display unit 110 and printing 120. The functional coupling of display unit 110 and 
20 printing 120 with the digitizing panel (not shown) produces a correspondence 
between displayed and printed graphics and touch actions. This allows system 
100 to receive and interpret input from the user. 

Printing 120 contains regions 120A and 120B. Gestures, or "moving 
25 touches" made in region 120A are interpreted as alphabetic characters, while 
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gestures made in region 120B are interpreted as numeric characters. Gestures 
made elsewhere are not interpreted as characters. 



There are many usability advantages realized by printing 120 over other 
5 prior art methods of facilitating gesture recognition. Some prior art systems lack 
printing 120, preferring instead to identify an area for gesture recognition on 
display unit 110, for example by displaying a graphical image similar to printing 
120. However, printing 120 identifies an area dedicated for gesture recognition, 
which aids the user in maintaining context between gesturing and selecting items, 
10 such as a soft activation key, on display unit 110. Another advantage of printing 

u 

^ 120 is the higher resolution achieved in printing 120 versus rendering a similar 
2; aide via the display unit. 



^ Unfortunately, these advantages (and more) are realized at the cost of 

a 

j;j 15 reducing the screen size of display unit 110 to accommodate printing 120. 

ri j 
yi 
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SUMMARY OF THE INVENTION 

Therefore, it would be advantageous to provide a method and system 
providing for the display of some information in the printed area used to facilitate 
gesture recognition while maintaining the many advantages of such a printed 
area. A further need exists for a hand held system that displays two independent 
information streams. A still further need exists for a system and method of 
displaying information using less power than a large, general purpose display. 

Embodiments of the present invention provide a method and system for 
displaying information in the printed area used to facilitate gesture recognition. 
Embodiments provide a method and system that display two independent 
information streams. Further embodiments provide for a method and system that 
display information using less power than a large display. Embodiments of the 
present invention provide these advantages and others not specifically mentioned 
above but described in the sections to follow. 

A method and system for displaying information in an area identified by 
permanent printing is disclosed. One embodiment of the present invention first 
displays information from a main program in a main display region. An interrupt 
signal from an event may be received. In response to the event, an independent 
information stream is displayed in a region identified by permanent printing. The 
permanent printing may be used to facilitate the recognition of stroke gestures. 
The user is able to interact with either program stream and its associated display 
information independent from the other program stream. 
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Another embodiment of the present invention is comprised of a single 
display unit, which has two separate display regions, one region being within the 
permanent printing region. 



5 In another embodiment of the present invention, each display region is 

composed of a separate display unit, with one display unit being within the 
permanent printing region. 



In yet another embodiment of the present invention, one display unit is 
10 configured to turn on in response to a signal that controls another display unit to 

CJ 

y) turn off. 



i :i 

m Another embodiment of the present invention provides for a method of 

E 1 

W displaying a keyboard image on a display region in the permanent printing region. 

5 

P 15 An alphanumeric character for input into the hand held computer system is 
1| determined from the correspondence of the keyboard image and touch 

Jrts, 

information. 

f" 5 ' 



In another embodiment of the present invention, information is displayed on 
20 a display region identified by permanent printing. The information allows a user to 
answer, conduct and end a telephone call without interrupting the use of a main 
display screen region. 
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In yet another embodiment, a short message, such as from the GSM Short 
Message Service (SMS), is received and displayed on a display screen region 
identified by permanent printing that may be used to facilitate gesture recognition. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is an illustration of a prior art hand held computer system. 

Figure 2A illustrates a hand held computer system according to an 
embodiment of the present invention. 

Figure 2B illustrates a hand held computer system with a representation of 
digital ink according to an embodiment of the present invention. 

Figure 3A illustrates a display unit assembly according to an embodiment of 
the present invention. 

Figure 3B illustrates a display unit assembly according to another 
embodiment of the present invention. 

Figure 4 illustrates a hand held computer system according to another 
embodiment of the present invention. 

Figure 5 is a flow diagram illustrating steps of a process for displaying 
information in a display region separate from a main display region and 
independent of the program flow represented in the main display region, 
according to an embodiment of the present invention. 

Figure 6 is a block diagram of a computer system, which may be used as a 
platform to implement embodiments of the present invention. 
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Figure 7 is a flow diagram illustrating steps of a process for displaying a 
keyboard image and selecting an alphanumeric character based on the 
correspondence of the displayed image and touch information, in accordance with 
an embodiment of the present invention. 

Figure 8 is a flow diagram illustrating steps of a process for turning on a 
second display unit and displaying clock information in response to turning off a 
main display region, in accordance with one embodiment of the present invention. 

Figure 9 is a flow diagram illustrating steps of a process for displaying 
digital ink in accordance with an embodiment of the present invention. 
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nFTAII FD DESCRIPTION OF THE INVENTION 

In the following detailed description of the present invention, a method and 
system for displaying information in a screen region identified by permanent 
printing that may be used to facilitate gesture recognition, numerous specific 
5 details are set forth in order to provide a thorough understanding of the present 
invention. However, it will be recognized by one skilled in the art that the present 
invention may be practiced without these specific details or with equivalents 
thereof. In other instances, well-known methods, procedures, components, and 
circuits have not been described in detail as not to unnecessarily obscure aspects 
F ,1 0 of the present invention. 

jjj NOTATION AND NOMENCLATURE 

f} Some portions of the detailed descriptions (e.g., processes 500, 700, 800, 

V 900) which follow are presented in terms of procedures, steps, logic blocks, 

|| 1 5 processing, and other symbolic representations of operations on data bits that can 

Pij 

be performed on computer memory. These descriptions and representations are 
Tl the means used by those skilled in the data processing arts to most effectively 
convey the substance of their work to others skilled in the art. A procedure, 
computer executed step, logic block, process, etc., is here, and generally, 
20 conceived to be a self-consistent sequence of steps or instructions leading to a 
desired result. The steps are those requiring physical manipulations of physical 
quantities. Usually, though not necessarily, these quantities take the form of 
electrical or magnetic signals capable of being stored, transferred, combined, 
compared, and otherwise manipulated in a computer system. It has proven 
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convenient at times, principally for reasons of common usage, to refer to these 
signals as bits, values, elements, symbols, characters, terms, numbers, or the like. 

It should be borne in mind, however, that all of these and similar terms are 
to be associated with the appropriate physical quantities and are merely 
convenient labels applied to these quantities. Unless specifically stated otherwise 
as apparent from the following discussions, it is appreciated that throughout the 
present invention, discussions utilizing terms such as "indexing" or "processing" or 
"computing" or "translating" or "calculating" or "determining" or "scrolling" or 
"displaying" or "recognizing" or "generating" or the like, refer to the action and 
processes of a computer system, or similar electronic computing device, that 
manipulates and transforms data represented as physical (electronic) quantities 
within the computer system's registers and memories into other data similarly 
represented as physical quantities within the computer system memories or 
registers or other such information storage, transmission or display devices. 

METHOD AND APPARATUS FOR DISPLAYING INFORMATION IN A DISPLAY 
SCREEN REGION IDENTIFIED BY PERMANENT PRINTING 
Embodiments of the present invention are described in the context of a 
hand held computer system. However, it is appreciated that the present invention 
may be utilized in other types of portable electronic devices that contain a 
permanent printing region on which a user may input data or control system 
function (e.g., beam data to another computer system) through gestures or 
handwriting recognition. 
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Figure 2A illustrates an exemplary hand held computer system 200. 
System 200 comprises a main display region 210, and a second display region 
220, located within a region of system 200 marked by permanent printing (also 
known as "silk screen region") 250. Printing 250 does not obscure display region 
220. 

Display region 220 may be used to display any type of textual and/or 
graphical information. As shown in Figure 2A, display region 220 displays an alert 
message, which is independent of, and does not interfere with, the information 
displayed in display region 210. In general, an alert message is designed to alert 
the user to an event that is not part of the current operation. 

Common alert messages include messages notifying the user of upcoming 
meetings and low battery states, or as shown in Figure 2A, the receipt of an 
incoming telephone call. It is appreciated that other types of alert messages are 
well suited to embodiments of the present invention. In prior art system 100, an 
alert message would typically be displayed on display unit 110, interrupting the 
information previously displayed on display unit 110. 

In one embodiment of the present invention, display region 220 is used to 
display clock information. 

In another embodiment of the present invention, display region 220 is used 
to display a short message received over a wireless communication channel. 
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In yet another embodiment of the present invention, display region 220 is 
used to display a keyboard image. 

It is appreciated that display region 220 is well suited to display very many 
5 other types of information, in accordance with embodiments of the present 

invention. Further examples include without limitation screen navigation toolbars, 
the running time of an audio/visual playback, a calculator display, flashing display 
region 220 as a whole or in part to attract a user's attention, displaying system 
status information and displaying advertising messages. 

Further, it is appreciated that information displayed in display region 220 
may have very many different relationships to information displayed (or not 
displayed) in display region 210, in accordance with embodiments of the present 
invention. 

Display region 210, display region 220 and printing 250 are functionally 
coupled to a digitizing panel (not shown) which is responsive to touch across the 
combined area of display region 210 and printing 250. The functional coupling of 
display region 210, display region 220 and printing 250 with the digitizing panel 
20 (not shown) produces a visual correspondence between displayed graphics and 
touch actions. This allows system 200 to receive and interpret touch input from the 
user interacting with display region 220 or printing 250. The term functional 
coupling is not intended to imply an assembly order or technique, as the 
correspondence of visual elements and touch-sensing elements can be achieved 
25 in a variety of well-known ways. 
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In a preferred embodiment of the present invention, printing 250 is used to 
facilitate gesture recognition, and contains regions 250A and 250B. Gestures, or 
"moving touches" made in region 250A are interpreted as alphabetic characters, 
while gestures made in region 250B are interpreted as numeric characters. 

It is appreciated, however, that in accordance with embodiments of the 
present invention, printing 250 may take other forms, including without limitation 
application program icons, system tool icons (e.g., a menu request icon), a 
wireless carrier logo, input shortcut icons (e.g., press here for ".com"), an icon to 
beam information to another computer system, an icon for switching languages of 
operation for the hand held device or images for controlling mobile telephone 
operation (e.g., "answer", "call", "end"). 

Figure 2B illustrates system 200 according to another embodiment of the 
present invention. Shown in Figure 2B are image 222 and optional stylus 224, in 
addition to elements previously described. It is appreciated that optional stylus 
224 is typically out of the plane of Figure 2B. 

Optional stylus 224 may be used to generate touch data to system 200 via 
the digitizer (not shown). It is appreciated that other forms of stylus 224 as well as 
user fingers are well suited to embodiments of the present invention. 

Responsive to a stroke gesture recognized by the digitizer (not shown) 
made in the region of printing 250, system 200 may display image 222. 
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Image 222 is representation of ink in display region 220, displayed "under" 
optional stylus 224. In a preferred embodiment of the present invention, image 
222 may be constructed in real time as the stroke gesture is made with optional 
stylus 224. That is, as each portion of the stroke gesture is made, a corresponding 
representation of that stroke is made until the final form of image 222 is achieved. 
It is appreciated that other well known techniques of establishing a time 
correspondence between image 222 and the stroke gesture are well suited to 
embodiments of the present invention. 

As depicted in Figure 2B t a user has made a stroke in region 250A of 
printing 250, which is interpreted by system 200 as the letter "n." In an exemplary 
implementation, the recognized letter "n" is displayed as the second "n" in the 
word "mankind" displayed in display region 210. It is appreciated that other well 
known uses of recognized gestures are well suited to embodiments of the present 
invention. 

Figure 3A illustrates a display module assembly 300 according to one 
embodiment of the present invention, as would typically be used in the 
construction of system 200. Assembly 300 comprises main display unit 320, 
permanent printing 250 to facilitate gesture recognition, display unit 310 located ii 
a region identified by printing 250, and digitizer panel 340. Digitizer panel 340 is 
responsive to the touch of an indicating device (not shown) across the combined 
area of display region 210 and printing 250. In a preferred embodiment, digitizer 
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panel 340 is a resistive panel. However, it is appreciated that other well-known 
digitizing panel types may be used with embodiments of the present invention. 

In a preferred embodiment, digitizer panel 340 is non-opaque, and is 
5 assembled to the front of display unit 310, main display unit 320 and printing 250 
so that all can be viewed through panel 340. It is appreciated that other possible 
arrangements of these components could be chosen. For example, printing 250 
could be applied to the top of digitizer panel 340. Likewise, an opaque digitizer 
panel 340, for example an electro-magnetic digitizer panel, could be placed 

_ 10 behind display unit 310, main display unit 320 and printing 250 so as not to 

G 

^ij obscure them. 

F = 

ris 

CI Fiqure 3B illustrates a display module assembly 360 according to another 

m embodiment of the present invention, as would typically be used in the 

E 

15 construction of system 200. Assembly 360 comprises main display unit 350, 

r ~ permanent printing 250 which may be used to facilitate gesture recognition and 

T[ digitizer panel 340. 

Display unit 350 contains a display region 355 located within the area of 
20 printing 250 and display region 351, which is outside of the region of printing 250. 

Thus, the single display unit 350 has two display regions - display region 
351 which is the "normal" or main display region outside of printing 250, 
corresponding to display region 210, and display region 355 located within the 
25 area of printing 250, corresponding to display region 220. 
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Figure 4 illustrates system 400 according to another embodiment of the 
present invention. System 400 comprises main display unit 320, digitizer panel 
340 (not shown in Figure 4 because it is transparent), and display unit 410 located 
within the area of printing 250. Display unit 410 is optimized to provide maximum 
contrast and brightness at minimum power consumption for the display of clock 
information. Display unit 410 is similar to the display unit in many digital 
wristwatches. Because display unit 410 has relatively few display elements, it can 
provide clock information at a lower level of power consumption than a more 
general purpose display unit. It is appreciated that other specialized display units 
are well suited to the present invention. Examples of other specialized display 
units include without limitation telephone displays, calculator displays and sound- 
power level bar graph displays. 

It is further appreciated that there is no requirement that display unit 320 
and display unit 410 employ the same display technology. For example, while 
display unit 320 might be a liquid crystal display (LCD) consisting of many rows 
and columns of display elements, display unit 410 might be an organic light 
emitting diode (OLED) type of display. These choices allow the designer of 
system 400 to optimize the display characteristics for varying requirements of 
display information and varying requirements of system operating conditions. For 
example, more light may be needed from display unit 410 than from display unit 
320 when display unit 410 is optimized for a telephone display. 
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In another embodiment of the present invention, display unit 410 can be 
configured to automatically turn on when main display unit 320 is turned off. 

Figure 5 illustrates the steps of a process 500 for displaying information on 
a display region, for example display region 220, independent of the information 
displayed on a main display region, for example display unit 210, in response to 
an event. 

In step 505, an interrupt is set up to provide the initial response to the event. 
The actual details of this step vary according to the specific details of the hardware 
and software, however the implementation of this step, given those specifics, is 
well known. 

In step 507, the main program is initialized and in step 510 display 
information for the main display region is generated. In step 515, the process 
displays program information to the main display region. The information may 
take a vast variety of forms, including without limitation datebook schedule 
information, a phone list, an interactive game, or as illustrated in Figure 2A on 
display region 210, an electronic book. 

In step 520 the process continues, looping back to step 510 to generate and 
ultimately display new display information. 
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It is understood that a typical program to generate and display information 
to the main display region will be significantly more complex. However, steps 510 
through 520 represent an exemplary process to illustrate the present invention. 

It is also well understood that responding to an event, also known as "taking 
an interrupt," will interrupt the sequential flow of a main program, and execute an 
interrupt routine. 

In step 550, just such an interrupt signal is received. The interrupt is 
generated from outside of the main program flow, for example from a radio- 
frequency telephone transceiver module. Another source could include a clock 
module. Still another example is a pager. The interrupt could even come from a 
different software module executing on the same system, for example, from a user 
input control routine which had been called previously by the main program. 

In response to the interrupt signal, a new sub-process is initiated at step 
555, in this example the sub process of controlling a telephone call is illustrated. 
The RF telephone module indicates to the system that an incoming phone call is 
available. 

In step 560, the user is given the opportunity to answer the call. The user is 
interrogated via display information sent to display region 220 as shown in Figure 
2A. The display of the question in display region 220 does not affect the display of 
program information in display region 210. By employing the well-known 
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technique of time slicing, user interaction with the program controlling main 
display region 210 is also not affected. 

The user actually makes the choice in step 565. In the example as shown 
in Figure 2A, touching the display over the letter "Y" makes the selection. It is 
appreciated that there are many other common techniques to capture a user 
response, including without limitation the use of hard buttons, thumb wheels and 
voice command. 

If the user chooses not to answer the call, process flow returns to the main 
program at step 510. If the user chooses to answer the call, process flow 
continues to step 570. 

Still referring to Figure 5, in step 570, the RF telephone module is 
commanded to answer the call. In step 575 the RF telephone module provides 
call status information to the system. 

In step 580, display region 220 is updated with call status information, for 
example the duration of the call. As with step 560, the display of new information 
to display region 220 does not affect display of program information in display 
region 210, nor does it affect user interaction with the main program. 

Part of the display information displayed in step 580 is a user interrogatory 
to end the phone call. Should the user select to end the phone call, step 585 will 
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direct the RF phone module to end the call, display information that the call has 
been ended, and transfer control back to the main program. 

Figure 6 illustrates circuitry of computer system 600, which may form a 
platform for a portion of hand held computer system 200. Computer system 600 
includes an address/data bus 650 for communicating information, a central 
processor 605 functionally coupled with the bus for processing information and 
instructions, a volatile memory 615 (e:g., random access memory RAM) coupled 
with the bus 650 for storing information and instructions for the central processor 
605 and a non-volatile memory 610 (e.g., read only memory ROM) coupled with 
the bus 650 for storing static information and instructions for the processor 605. 
Computer system 600 also optionally includes a changeable, non-volatile memory 
620 (e.g., flash) for storing information and instructions for the central processor 
605 which can be updated after the manufacture of system 600. 

Computer system 600 also optionally includes a data storage device 635 
(e.g., a Secure Digital memory card) coupled with the bus 650 for storing 
information and instructions. 

Also included in computer system 600 of Figure 6 is an optional 
alphanumeric input device 630. Device 630 can communicate information and 
command selections to the central processor 600. Device 630 may take the form 
of a touch sensitive digitizer panel. 
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The display unit (either display unit 320 or display unit display unit 350) 
utilized with the computer system 600 may be a liquid crystal display (LCD) 
device, cathode ray tube (CRT), field emission device (FED, also called flat panel 
CRT), light emitting diode (LED), plasma display device, electro-luminescent 
display, electronic paper or other display device suitable for creating graphic 
images and alphanumeric characters recognizable to the user. 

System 600 optionally includes a second display unit 310. Display unit 310 
may be a liquid crystal display (LCD) device, cathode ray tube (CRT), field 
emission device (FED, also called flat panel CRT), light emitting diode (LED), 
plasma display device, electro-luminescent display, electronic paper or other 
display device suitable for creating graphic images and alphanumeric characters 
recognizable to the user. In particular, display unit 310 may use a different display 
technology than display unit 320. 

System 600 optionally includes a radio frequency module 660, which may 
implement a mobile telephone, a pager, or a digital data link. Optional signal 
input/output communication device 640 is also coupled to bus 650. 

Figure 7 illustrates the steps of a process 700 for displaying a keyboard 
image and sensing an alphanumeric user selection based on the correspondence 
of a user touch to the displayed keyboard image according to an embodiment of 
the present invention. 
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Process 700 is typically initiated as a subroutine in response to a main 
program request for alphanumeric input. Such requests are common in hand held 
computer systems, occurring, for example, when the user desires to put a name 
into an address book application. 

5 

In step 710, a keyboard image is displayed in display region 220. The 
display image may be general purpose or it may be specialized for the particular 
application running on the hand held computer system. One such specialized 
keyboard image may be for a scientific calculator, which displays numbers and 
1 0 keys for trigonometric functions. Another specialized keyboard image may be for 
a phone, including number keys for dialing, "send" and "end" keys for control of 
the phone call, and volume control keys. 

In step 720, control is transferred to the digitizer control function, which is a 
15 standard function of such hand held computer systems. 

In step 730, digitizer information is returned from the digitizer control 
routine. Such digitizer information generally takes the form of an X, Y coordinate 
pair. It is appreciated that many other methods of describing a touch event are 
20 possible. 

In step 740, the digitizer information is associated with the keyboard image. 
For example, if the keyboard image was a QWERTY keyboard, the pair 1 , 3 might 
indicate the letter "e." The chosen symbol is then encoded into the character code 
25 used by the hand held computer system. 
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In step 745, the keyboard display is optionally removed from the display 
screen region. 

In step 750, the character code is returned to the calling routine, as well as 
program flow. 

Figure 8 depicts the steps of a process 800 to turn off the main display unit, 
for example display unit 320, and automatically turn on a second display unit. 

In step 810, the main display unit 320 is turned off. This is usually 
controlled by writing a control bit to a display control register. It is appreciated that 
there are many well known methods to accomplish control of the display, including 
both hardware and software based implementations, that are well suited to 
embodiments of the present invention. 

Responsive to this signal, second display unit 410 is commanded on in step 
820. Again, it is appreciated that there are many well known methods to 
accomplish control of this display as well. 

Finally, in step 830, display information is sent to the second display unit 
410, in this example the information facilitating a clock display. 
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In Figure 9, process 900 of displaying ink under gestures in a region 
identified by permanent printing according to an embodiment of the present 
invention is described. 

5 Process 900 is typically initiated as a subroutine in response to a main 

program request for alphanumeric input. Such requests are common in hand held 
computer systems, occurring, for example, when the user desires to put a name 
into an address book application. When the main program determines that 
gestures are being made in the region of printing 250, that touch information is 
r] 1 0 sent to process 900. 

a) 

fj] in step 930, the touch information is received by process 900. 

fn 
r A 

in step 940, the touch information is mapped to display region 220, such 
D> 1 5 that a representation of ink appears under the location touched by the user. 

Wi 
P 

p In step 950, control is returned from process 900 to the calling routine. 

The preferred embodiment of the present invention a system and method 
20 for displaying information on a display screen region identified by permanent 
printing is thus described. While the present invention has been described in 
particular embodiments, it should be appreciated that the present invention should 
not be construed as limited by such embodiments, but rather construed according 
to the below claims. 
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